Group Integration Techniques in Pattern Analysis - A Kernel View

Pattern analysis deals with the problem of characterizing and analyzing relations in data in an automated way. A quite important issue during the design process of such algorithms is the incorporation of prior knownledge; knowledge that is related to all information about the problem available in addition to the training data. This thesis is about a certain kind of prior knowledge: we assume to know that the data does not change its meaning under certain transformations, that is, the pattern recognition process has to be ’invariant’ under these transformations. In statistical pattern recognition the data is typically given in vectorial form. In this work we assume that the transformations affect this vectorial data just by a linear group-like transformation. This comprises, for example, translations or rotations of the object under consideration. Typically, the recognition process can be expressed as classification or regression functions based on the before mentioned vectorial data. Kernel-based models of these functions arose in the mid-1990s. This new approach enabled researchers to analyze non-linear relations with the effectiveness that have been previously reserved for linear algorithms. From a computational and a conceptual, mathematical point of view the kernel-based pattern analysis algorithms are as efficient and as well founded as linear ones, whereas problems like local minima and overfitting that were typical of previous non-linear methods have been overcome. In this work we will use the conceptual ease of kernel-based models to build a theoretical well-founded framework of invariance in pattern analysis. The linear nature will help us to provide, in a surprisingly simple way, optimality statements that are closely related with the principle of group integration having its origin in classical invariant theory. In the context of pattern recognition group integration has been playing a major role from the beginning while in most cases the use is rather implicit. This work is divided into a theoretical and a practical part. In the theoretical part we establish the beforementioned framework that connects invariance theory with kernelbased methods. The considerations include a precise formulation of the notion of invariance and its manifestation in kernel spaces. As a main result a representer theorem is proven, showing that any solution of a learning problem with certain invariance constraints makes use of the principle of group integration. Furthermore, we provide characterizations and interpretations of the occurring kernels, we mention kernels with different concepts of invariance and show connections to related work like Volterra theory. The practical part concerns on the one hand the feature extraction process and on the other hand the kernel framework itself. We propose features that are based on group integration and evaluate them with two classification tasks, classification of surface models and protein structures. Secondly, we propose a rotation-invariant object detection concept based on the generalized Hough transform. Thereby, we model the mapping from the local appearance patches onto the spatial probability density by a kernel machine.